[Segment Replication] Refactor RemoteStoreReplicationSource (#8767)#9076
Merged
dreamer-89 merged 3 commits intoAug 3, 2023
Merged
Conversation
dreamer-89
requested review from
Bukhtawar,
CEHENKLE,
Rishikesh1159,
VachaShah,
anasalkouz,
andrross,
dblock,
dbwiddis,
gbbafna,
kartg,
kotwanikunal,
mch2,
nknize,
owaiskazi19,
reta,
ryanbogan,
sachinpkale,
saratvemulapalli,
setiah,
shwetathareja,
sohami and
tlfeng
as code owners
Contributor
Gradle Check (Jenkins) Run Completed with:
|
…ch-project#8767) * [Segment Replication] Refactor remote replication source Signed-off-by: Suraj Singh <surajrider@gmail.com> * Unit test updates Signed-off-by: Suraj Singh <surajrider@gmail.com> * Self review Signed-off-by: Suraj Singh <surajrider@gmail.com> * Self review Signed-off-by: Suraj Singh <surajrider@gmail.com> * Segregate shard level tests for node to node and remote store segment replication Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix failing unit tests Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix failing UT Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix failing UT Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix more unit tests Signed-off-by: Suraj Singh <surajrider@gmail.com> * Improve RemoteStoreReplicationSourceTests, remove unnecessary mocks and use actual failures for failure/exception use cases Signed-off-by: Suraj Singh <surajrider@gmail.com> * Spotless check fix Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments Signed-off-by: Suraj Singh <surajrider@gmail.com> * Ignore files already in store while computing segment file diff with primary Signed-off-by: Suraj Singh <surajrider@gmail.com> * Spotless fix Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix failing UT Signed-off-by: Suraj Singh <surajrider@gmail.com> * Spotless fix Signed-off-by: Suraj Singh <surajrider@gmail.com> * Move read/writes from IndexInput/Output to RemoteSegmentMetadata Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review commnt Signed-off-by: Suraj Singh <surajrider@gmail.com> * Update recovery flow to perform commits during recovery Signed-off-by: Suraj Singh <surajrider@gmail.com> * Remove un-necessary char Signed-off-by: Suraj Singh <surajrider@gmail.com> * Address review comments Signed-off-by: Suraj Singh <surajrider@gmail.com> * Update comment nit-pick Signed-off-by: Suraj Singh <surajrider@gmail.com> * Remove deletion logic causing read issues due to deleted segments_N Signed-off-by: Suraj Singh <surajrider@gmail.com> * Spotless fix Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix unit tests Signed-off-by: Suraj Singh <surajrider@gmail.com> --------- Signed-off-by: Suraj Singh <surajrider@gmail.com>
Signed-off-by: Suraj Singh <surajrider@gmail.com>
dreamer-89
force-pushed
the
backport/backport-8767-to-2.x
branch
from
2e37b6b to
fbed447
Compare
mch2
approved these changes
Aug 3, 2023
Contributor
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Suraj Singh <surajrider@gmail.com>
6 tasks
Contributor
|
Compatibility status: |
Contributor
|
Compatibility status: |
Contributor
Gradle Check (Jenkins) Run Completed with:
|
Poojita-Raj
approved these changes
Aug 3, 2023
Codecov Report
@@ Coverage Diff @@
## 2.x #9076 +/- ##
============================================
+ Coverage 70.72% 70.86% +0.13%
- Complexity 57372 57497 +125
============================================
Files 4754 4754
Lines 271625 271647 +22
Branches 40066 40067 +1
============================================
+ Hits 192102 192494 +392
+ Misses 62972 62655 -317
+ Partials 16551 16498 -53
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Manual backport of #8767 to 2.x
Note: This needs to be merged in after as this PR contain changes on top of below two PRs. Keeping in draft until below are backported.